Performing security testing on a computer system might involve exercising parts of the functionality of the computer system and evaluating whether an instance of security vulnerability exists. For example, if a computer system is supposed to be accessible only to authorized persons and is supposed to block unauthorized persons, a simple test might be for a tester to access the computer system and at a login screen that asks for a user name and a password, type in a known invalid name, such as “nobody” and a known invalid password such as leaving the password field blank and then submitting that as a login. If the computer system responds by allowing the tester past the login screen as if the tester were an authorized person, that indicates that the computer system has a security vulnerability. If the computer system responds by displaying a message such as “Unrecognized user name or password” and remains at the login screen, that may indicate that the computer system might not have that particular vulnerability.
This is, of course, an extremely simple test and fully testing a computer system for moderate complexity of vulnerabilities can be quite involved. For example, a computer system might have a vulnerability that is only noticed if a tester inputs an unexpected string into a field, such as entering “; DROP TABLE users” into a field that is used to enter a user name. The computer system might have many different locations in an application that ask for user name and it might be that some of those correctly respond by refusing to process the improper input while others would process the improper input and perform actions that the designers of the computer system assumed would not be allowed to ordinary users.
A typical computer system might be executing a complex application, such as a web banking application that handles information display, transaction generation and funds transfers, an e-commerce application that handles product display, online shopping, purchasing and shipping, or other complex systems. With such systems, it can be useful to receive vulnerability reports from a great many of testers who might be operating independently of other testers. These testers might be security professionals testing a target computer system on behalf of a security company that provides compensation or rewards to testers who submit vulnerability reports.
One problem with providing compensation or rewards to testers who submit vulnerability reports is that the award is typically limited to the first submitter and multiple independent testers might report the same vulnerability. An evaluator might manually read all of the incoming vulnerability reports and determine that two or more vulnerability reports are for the same vulnerability and then determine which tester was first to report the vulnerability. This is a considerable undertaking for a large system under test and can sometimes be impractical. It is not a simple matter to do a word-by-word comparison of the vulnerability reports by multiple submitters, because not all testers would use the same language or words.